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Abstract 

In this paper, we apply Devroye inequality to study various statis- 
tical estimators and fluctuations of observables for processes. Most of 
these observables are suggested by dynamical systems. These appli- 
cations concern the co-variance function, the integrated periodogram, 
the correlation dimension, the kernel density estimator, the speed of 
convergence of the empirical measure, the shadowing property and 
the almost-sure central limit theorem. We proved in 5 that Devroye 
inequality holds for a class of non-uniformly hyperbolic dynamical sys- 
tems introduced in jl8| . In the second appendix we prove that, if 
the decay of correlations holds with a common rate for all pairs of 
functions, then it holds uniformly in the function spaces. In the last 
appendix we prove that for the subclass of one-dimensional systems 
studied in 18 the density of the absolutely continuous invariant mea- 
sure belongs to a Besov space. 

Keywords. Integrated periodogram, correlation dimension, kernel 
density estimator, empirical measure, shadowing, almost-sure central 
limit theorem. 



^Acknowledgments. JRC and PC acknowledge the CIC for its warm hospitality, 
Cuernavaca, Mexico, where part of this work has been done, as well as the Institut de 
Mathematiques de Bourgogne in Dijon. BS acknowledges the kind hospitality of CPhT at 
Ecole Polytechnique. The authors acknowledge the anonymous referee for a very careful 
reading of the paper. 



1 



Contents 

1 Introduction and set-upl 2 

2 Covariance functioni 4 

3 Integrated periodogrami 5 

4 Correlation dimensioni 10 

5 Empirical measur 1 11 

6 Kernel density estimation for ID mapsi 14 

7 Shadowing and mismatchi 16 

8 Almost-sure central limit theoreml 18 
A About a trigonometric serie 

2 22 

B On the uniform decay of correlations! 23 



C A property of the density of the inyariant measure for a class of ID maps! 25 

1 Introduction and set-up 

Assume one has a finite sample xi,. . . ,Xn of a stationary ergodic process 
taking values in M*^. If we consider an empirical estimator (or an observable) 
K{xi, . . . , Xn) of some statistical properties of the process, we basically wish 
to determine its fluctuations and its convergence properties, as n grows. In 
Statistician's terminology, we aim to study the consistency of the estimator 
K{xi, . . . , Xn) and be able to build confidence intervals. 

As we shall see in the sequel with various examples, many interesting 
estimators have a complicated dependence on the sample. In particular 
they are not of the form {u{xi) + • • • + u{xn))/n, for some function u, or 
cannot be well approximated by such time-averages for which the Central 
Limit Theorem may apply. 

The aim of this paper is to apply what we call Devroye inequality |S], 
see the definition below, to estimate the variance for a general class of esti- 
mators K(xi, . . . ,Xn)- For some of them we will further require some weak 
conditions on the auto-covariance function for functionals of the process. 

Our applications concern the empirical auto-covariance function, the in- 
tegrated periodogram, the correlation dimension, the kernel density estima- 
tion of the density of the invariant measure, shadowing properties, the speed 
of convergence of the empirical measure toward the invariant measure, and 
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the almost-sure central limit theorem. Some of these estimators were stud- 
ied in [H] in the context of piece-wise expanding maps on the interval for 
which a stronger inequality than Devroye inequality holds. 

We shall formulate the results as much as possible in an abstract setting 
in order to see more clearly what is needed to prove them. As we showed 
in [S], a class of non-uniformly hyperbolic dynamical systems introduced by 
L.-S. Young ^S] fits this framework. 

Let (rJ,53,P) be a probability space and (Xk) be a stationary ergodic 
sequence of random variables assuming values in W^. 

We will denote the expectation with respect to P by E, and by /i the 
common distribution of the X^s. We will assume that the X^'s are almost- 
surely bounded, i.e. there exists a positive constant A such that 

\\Xk\\ < A F - almost surely. (1) 

Let K he a real- valued function on (M*^)". We will say that K is sep- 
arately T^-Holder in all its variables, if for any 1 < i < n, the following 
quantities are finite 

Lj = Lj{K) := sup sup (2) 

Xl,X2,...,Xj-l,Xj,Xj + -l,...,X„ XjJ^Xj 

\K(^Xi , . . . , , Xj , , . . . , Xji) K(^Xi , . . . , Xj—\, Xj , Xj-i-i , . . . , Xn)\ 

\\Xj — XjW'^ 

We now define what we mean by saying that the process {Xk) satisfies 
Devroye inequality. 

Definition 1.1 (Devroye inequality for the variance). We will say that 
the process (Xi^) satisfies Devroye inequality if, for r] g]0, 1], there exists a 
constant D = D[r]) > such that for any integer n > 1 and for any real- 
valued separately r]- Holder function K on (M*^)", we have 

n 

var(i^) = ¥.{{K- E(i^))^) <D . (3) 

j=i 

For the case of Dynamical Systems, is the phase space on which acts 
a measurable transformation /. We assume that an /-invariant ergodic 
measure /i is given. One can define a stochastic process Xj^{x) = f^~^{x) 
where x is randomly chosen according to fi. We are interested in observables 
of the form K{Xi, = K{x, /(x), . . . , f^-Hx)). 

One can ask whether there are processes satisfying Devroye inequality. 
Indeed, a large class of dynamical systems satisfy Devroye inequality, as we 
proved in [3]. Let us recall that this class contains families of piece- wise 
hyperbolic maps, like the Lozi maps; scattering billiards, like the planar 
periodic Lorentz gas; quadratic and Henon maps (for parameter sets with 
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positive Lebesgue measure). Let us also briefly mention that such dynamical 
systems admit an SRB-measure, enjoy exponential decay of correlations and 
a central limit theorem for Holder continuous observables. Notice that in the 
sequel we will only need very slow decays of correlations, e.g., C{£) ~ l/Vi 
for the integrated periodogram or absolute summability for, e.g., the almost- 
sure central limit theorem. 



2 Covariance function 



Recall that the auto-covariance of a real-valued, square-integrable, function 
u on M.'^ is defined by 

C(n) = Cu{n) := E{u{Xi)u{Xn)) - (E(7x(Xi)))2 . (4) 

An empirical estimator of the auto-covariance is given by 

1 ^ 

Ck{n) = - '^u{Xj)u{Xj+n) 
i=i 

It follows at once from Birkhoff 's ergodic Theorem that 
C(n) = lim Ck{n) P — almost surely. 

fc^oo 

Theorem 2.1. Let u be a real-valued r]- Holder function on with Holder 
constant denoted by L^. Then, for all integers k, n, we have 

n + k D^Lt 
+ 




E[{Ck{n)-C{n)f)<16DLtA'^ , 
Proof. We have the following identity : 

E ((Ckin) - C{n)f 



E ((4(n) - nCk{n))f) + (E((7fc(n)) - C{n))f = 

var(C'fe(n)) + |^var(i ^ j • 

The first term is estimated using Devroye inequality Q and assuming, with- 
out loss of generality, that n(0) = 0. We obtain the upper-bound of inde- 
pendent interest 

var(4(n))<16Z)L^^2r,!H_fc_ 

The second term is easily estimated using again Devroye inequality. This 
leads immediately to the above estimate. ■ 



Remark. For the study of ^/-statistics of functionals of a- and /3-mixing 
process we refer the interested reader to [2] and references therein. 
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3 Integrated periodogram 



We recall (see 0) that if u is a real- valued function the raw periodogram 
(of order n) of the process (n(Xfc)) is the function 



n 



J^e-^- (n(X,)-E(n(Xi))) 



(5) 



where uj E [0, 27r]. The spectral distribution function of order n (integral of 
the raw periodogram of order n) is given by 



Jn{uj) = / I„(s) ds . 



(6) 



Prom a practical point of view, it is worth defining the empirical spectral 
distribution function of order n as follows: 



J„(cj) 



-IJS 



^ e=i / 



ds . 



In this section we will make the following assumption. 

Hypothesis 3.1. The function u is "q-Holder continuous and its auto- 
covariance function C{£) = Cu{t) satisfies 



< oo 



(where C{£) is defined at ^). 

Let C{iv) be the Fourier cosine transform of the auto-covariance function, 
namely 

oo 

C{u;) =^cos{iuk) C{k + 1). 

k=0 

We will denote by J (to) the integral of the following quantity 

sm{ujk) 



J{uj) = / (2(7(s) - C(l)) ds = C(l) w + 2 ^ 



C{k + l). (7) 



fc=i 



We will use the following convenient quantity: 



n — 1 



\C{k + l)\ 
k 



Observe that J{uj + 27r) = J{uj) + 27rC(l) = J{uj) + J(27r). In order to 
estimate J, it is therefore enough to restrict to the interval [0, 27r]. 
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Theorem 3.1. There exists a positive constant T such that for any function 
u satisfying Huvothesis Vj . li and any n > 1, we have: 



E 



sup I Jn(w) — </(w) 

cJ6[0,27r] 



< r mi {N 

N>1 



C(l)2 + D^2^ L^(l + logn) 



+ A 



n 



+ 



+ A 



N 



Remark. If A„ < const /n, then 

E 



sup |J„H - JMl)' I < 0(l)ii±l^^n)^ _ 

In particular, if the auto-covariance is absolutely summable, then A„ < 
const /n. 

For convergence results in distribution sense of the raw periodogram for 
a class of maps on the interval we refer to jl2j . 

This theorem is the consequence of two propositions. 

Proposition 3.1. For any function u satisfying Hypothesis \ and any 
n > 1, we have: 

E ( ( sup IJ^H- JMl) ) < 



inf 

N> 



Proof. Let 



Qn= sup I J„(u;) - J(cj) 

aje[0,27r] 



(8) 



Let N be an integer and define the sequence of numbers {ujp) by uop = 2'Kp/N 
for p = 0, . . . , A^. It follows at once from the monotonicity of J and Jn (since 
they are integrals of non-negative functions) that 



(5„ < max sup \Jn{iOp+l) - J{t^p)\, sup \Jn{iOp) - J{uJp+i)\\ . 

\0<p<N-l 0<p<N-l j 

We now have 

Qn < sup \Jn{uJp) - J{LOp)\ + SUp \J{uJp) - J(tJp+l)| . 

0<p<N 0<p<N-l 

Now using definition 0, we get after an easy computation that for all 
j5 = 0, . . . , iV - 1 



\J{LOp) - JK+i)| < 2Tr + 



(9) 



6 



It follows that 

( C{1) 



Qn<Qn + 2^ + AiV j (10) 

where 

(5„ = sup \Jn{uJp) - J{uJp)\ . 
0<p<N 

We obviously have 

N 

E(qI) < ^ E( ( J(a;p) - Ucop))' ) . (11) 
p=0 

We now estimate each term E( {J(u)p) — Jn('^p))^ )• Observe that for any uj 
we have 

E( {J{i0) - Jn{uj)f ) = E( {Jn{uj) - nJn{^))f ) + {Wnioj) " J {^))f ■ 

We have also from the definition of J„ 



n ^-^ 



+ - E ' ■ r - nniXi))iuiX,) - E(n(Xi)) 



5^(n(X,)-E(t.(Xi))^ 



+ ^ E (^(^.O - IE(^(^i))(n(X,) - E(n(Xi)) . (12) 

Using this formula and ((T)), an easy computation leads to 

(E(J„(u;)) - J{uj))f < Al . (13) 
We now apply Devroye inequality to Jni^) in the form (|12() and get 

E( (J„(.) - E(J„(-)))^ ) < + ^ + . (14) 

n 

Using (0, (CH) and (dJ, it follows that 

E(Q^) < (AT + 1) ^(^l±l±^^^I^A-m + A^) . (15) 
This completes the proof. ■ 
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Proposition 3.2. There exists a positive constant S such that for any func- 
tion u satisfying Huvothesis \3.1\. and any n>\, we have 



E I ( sup I J„(cj) - J„(u;)| 

■<^e[0,27r] 



< S inf UiV + 1) 

Af>l 



C{lf + DA'^Lt , .2 



n 



+ 



N 



+ A 



N 



The proof is rather similar to the previous one. 
Proof. Let 

Rn= sup I Jn(w) - Jn(w)| . 
a;G[0,27r] 



(16) 



Let be an integer and define as before the sequence of numbers (ujp) by 
Up = 27Tp/N for p = 0, . . . , N . It follows at once from the monotonicity of 
J„ and Jn that 



i?n < max sup \Jn{uJp+l) - Jn{uJp)\, SUp | J„ (Wp) - Jn (Wp+l) | 

\0<p<N-l 0<p<N-l 



We now have 



where 



Rn<Rn+ sup \Jn{uJp) - Jn{i^p+l)\ 
0<p<N-l 



Rn= sup \Jn{uJp) - Jni^^p) 
0<p<N 



(17) 



Now we have the estimate 



E sup {Jn{uJp) - JniuJp+l)f < 

V0<p<Af-l J 



6E sup {Jn{ujp) - J{ujp)f ] + 3 sup {J (ujp) - J {Ujp+l))'^ . 

\0<p<N J 0<p<N-l 

Using Proposition 13. II to estimate the first term and © for the second one, 
we obtain 

E sup (Jn(Wp) - Jn(Wp+l))^ < 

yo<p<Af~i J 

' • (18) 



12(iV + 1) ('1!L!^a2, ^ ^ go^2 f ^ + 



n 



N 
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We obviously have 



N-l 



p=0 



(19) 



We now have to esthnate each term E yyJn{i^p) — Jn{^p))'^j- Observe that 
for any uj 

E ((J„,(a;) - Jniio)f) = var (j„(w) - J„(a;)) + (e(J„(u;) - J„,(w))' ' 



Let Sn ■= Yll=i'^i-^j)- ^ simple computation yields 



Jn{L0) - Jn{uj) = CU — - E(^.(Xi)) 



n 



(20) 



An easy computation leads to 



n 



sin ((j - €)uj) 
j-i 



An easy computation using Lemma lA.ll shows that there is a constant ci > 
such that for all integer n 



sup 

i^e[0,27r] 



1 ^ sin((j - £)uj) 



^--2^ 7 / • |E( (--E(^(Xi)) 



n 



< 



ci 



( Cjl) 
\ n 



+ A, 



Similarly, there exists a constant C2 > such that 



sup 

w6[0,27r] 



sin((j - i)uj) 



r=l 1=1 jj^l 

Combining these two estimates, one gets 



<C2 (^ + A„ 



sup E( J„(w) - J„(c.;)) < C3 { + A„ 

<^6[0,27r] ^ / V n 



( C{1) 



(21) 



where C3 > is a constant (independent of n). We now apply Devroye 
inequality to Jn{^) — Jn{^) using (PU]) and Lemma fA.ll We easily obtain 
the estimate 

sup var Jnioj) - Jn{uj) < ■ (22) 

a;e[0,27r] ^ / n 



It follows that 



^2 



C(l) , \2 , cDA^^Lt 



E(i?„) <N\c3 + A„ + 



n I n 



The Proposition follows by combining this estimate with (|18j) . 



Theorem 13. II is proved by combining Propositions I3.ll and 13.21 



4 Correlation dimension 

We recall that the correlation dimension = dc{^) of the measure ji (recall 
that jjL is the common distribution of the X^'s) is defined by 

^.^ log//i(i?(x\e)) d^{x') 
eJ,o loge^i 

provided the limit exists (where B(x',e) is the ball of centre x' and radius 
e). In practice one determines for large n the power-law behaviour in e of 
Kn,e{^' /(^)' • • • ' /""H^;)) where 

K^^^{xi, ...,Xn) = ^^^{e- d{xi,Xj)) 

and "i? is the Heaviside function (i.e., the characteristic function of M+). It 
is known that (see e.g. |14j ) 



lim KAx,fix),...,r-\x)) = / KBix',e)) d^x') 

n— >oo 



for /i-almost all x and every continuity point of the non-increasing function 
e ^ J fi{B{y,e)) dfi{y). 

To proceed we need to replace K^^(xi, . . . , x^) by a component-wise 
Lipschitz function. For any real-valued Lipschitz function (j), define the 
sequence of component-wise Lipschitz functions 



(23) 
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Theorem 4.1. For any real-valued Lipschitz function (j), for any < 77 < 1, 
there exists a constant C = C{r]) > such that for any e > and any integer 
n, we have 

var(i^;t,J < . (24) 

The proof is a direct appUcation of Devroye inequaUty (jHJ . 

Several functions (j) are used in the literature. A simple one is given by 

r fory<-i 

My) = \ \ + y for - i < y < i 

[1 for y > i • 
One verifies easily that for all y G R 

??(l-2y)<<Ao(l-y)<^?(l-y/2). (25) 
This implies immediately 

<e/2(^l> • • • ,^n) < Kt%{xi, ...,Xn)< <2.(^1, • • • , ^n) (26) 

for all xi, . . . , e > and n > 1. It follows that, when d^ > 0, we have 
<,(x,/(x),...,r-i(x))^e'^^ ase^O 

is equivalent to 

<o (x, fix), f^-\x)) ^e"^ as e ^ . 

Requiring that the typical value is smaller than the size of fluctuations (stan- 
dard deviations) leads to e"^" > \/{e^^/n). In other words 

In some iid cases, the optimal estimate has been obtained in |llj . 

5 Empirical measure 

We recall that the empirical measure of a sample Xi , . . . , Xn is a random 
measure on M'^ defined by 

1 

n ^-^ ■' 

where 6 denotes the Dirac measure. We recall that from Birkhoff 's ergodic 
theorem, almost-surely this sequence of random measures weakly converges 
to the common distribution // of the X^s. It is natural to ask for the speed 
of this convergence. This of course depends on the distance chosen on the set 
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of probability measures. We will consider the Kantorovich distance defined 
for two probability measures /ii and ^2 on by 

K{m,fi2) = sup / g{x) d{ni - H2){x) (27) 

where ^ denotes the set of real- valued Lipschitz functions on M*^ with Lip- 
schitz constant at most one. 

We now state the theorems of this section. 

Theorem 5.1. By Devroye inequality ^ we have, for all n > 1, 

m 



n 



The proof follows at once from Devroye inequality (jSJ using the following 
separately Lipschitz function of n variables 



K{xi, ...,Xn) = sup 



^E5(x,)-E(<7) 



n 



To get a probability estimate based on this result one needs to give 
an upper-bound for E(k(£^„, //)). The bound we are so far able to obtain in 
dimension larger 1 is too pessimistic. We explain below how to obtain a more 
satisfactory bound in dimension 1. We will require the following property 
for the auto-covariance. We will denote by the ?7-Holder constant of u 
(which is bounded by 0{l)Li{u)). 

Hypothesis 5.1. For any r] g]0, 1] there is a constant > such that the 
auto-covariance Cu{i) of any r]-Hdlder continuous function u satisfies 



£=1 

This leads to the following theorem. 

Theorem 5.2. Assume that the process {X^) takes values in M and that the 
auto-covariance of rj-Holder continuous functions satisfies Hypothesis \5.1\ 
Then, for any r] g]0, 1], there exists a positive constant a{r]) such that for 
all t > and n> 1, we have 



^1/(2(1+,?)) / - 



Remark. If aij]) behaves like 1 /rj as rj tends to zero, then one can optimize 
by taking rj = 1/logn. 
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Proof. The theorem of Dall'Aglio 7^ states that 



JR 



rA 1 "~1 

{£n,^Ji)= / dt -y^i){t - Xk) - F^{t) 
J -A 



where -F^(i) is the distribution function of ^. 

We wish to estimate the Kantorovich distance between the empirical 
measure £n and fj, (the common distribution of the X^'s). In this case we 
have 

K[£n,^) = I 

I A n 

k=0 

since we assumed from the very beginning that \\Xk\\ < A P-almost-surely, 
and 1? denotes the Heaviside function. 

In order to use the decay of correlations, we replace the Heaviside func- 
tion by a Holder continuous function gs parametrised by a positive 6 and 
defined by 

r if s<-6 

gg{s) = } l + s/6 if -6 <s<0 
[ I if s > . 

We immediately obtain 

pA 

Ki£n,fJ-) <5+ dt -V] gsit - Xk) - F^{t) 

We have 

E(K(f„, /i)) <<5 + e / dt -Y, 9s{t - Xk) - K{gs{t - Xi)) 



(28) 



k=0 



+ 



25 



dtE\gs{t-Xi)-§{t-Xi)\ < 

J-A 
( l-A 1 

+ E / dt - Y^g^(t-Xk)-E{gi(t-X{)) 



Using Cauchy-Schwarz inequality as in [HI, one is led to use the decay of 
auto-covariance of the functions 55 (i — •). Using Hvpothesis 15.11 we get 

E(K(f„,/x)) < 25^^}^^ 



5''^y/n 



Using Chebychev inequality, the above estimate with 5 = n 1/2(1+'?) and 
Theorem 15. II we get the theorem. ■ 
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For the application to dynamical systems satisfying Devroye inequality 
(see jS]), we need moreover to verify Hvpothesis 15.11 It is often proved, 
see e.g. |18j . that the auto-covariance of observables belonging to a Banach 
space have a common upper bound for their rate of decay. It turns out 
that this implies a uniform rate of decay for all functions of norm less than 
or equal to one, this is the content of Theorem IB. II proved in Appendix 
IbI So, if this decay is summable then Theorem 15.21 holds. For the systems 
studied in ^H], Hvpothesis 15.11 can be deduced using the estimates provided 
by approximations #1 and #2 and point 4.2. appearing in that paper. 



6 Kernel density estimation for ID maps 



In this section we assume that d = 1, namely that the process (Xj.) takes 
values in a bounded interval of M. Moreover we assume that the common 
distribution fj, of the X^s is absolutely continuous (with respect to Lebesgue 
measure) and denote by ^ its density. We consider the random empirical 
densities (hn) defined by 

1 

hn{Xi, . . .,Xn; s) = y^^'ipiis - Xj)/an) 

nar ^-^ 



where a„ is a positive sequence converging to and such that na„ converges 
to +00, and '0 (the kernel) is a bounded, non-negative, Lipschitz continuous 
function with compact support whose integral equals 1. We are interested 
in the convergence of these empirical densities to the density $ of the 
common distribution of the X^'s. 



Theorem 6.1. Assume that the probability density <I> satisfies 



J |$(s) -^>(s-y)| ds<C 



(29) 



for some C > 0, t > and any y G M. Suppose also that Hvvothesis \5.1\ 
holds. Then, for any rj g]0, 1], for any tp as above, there exists a constant 
C = C'{rj^il)) > such that for any integer n and for any t > C'{an + 
l/{\^an"^^)), we have 

c 



|/i„(Xi,...,X„;s)-$(s)| ds>t] < 
Proof. We define the functions 



t'^nol^ 



K{xi, ...,Xn) 



1 " 

Y.H{s-Xj)/an)-<^{s) 



ds . 
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It is easy to verify that the Holder constants of this ?7-H61der continuous 
function satisfy 

max Lj < fr- • 

Hence, using Devroye inequahty we immediately obtain 



var(K) < 



na 



2r} 



The theorem will follow using this and Chebychev inequality provided we 
have an upper bound for E(i^). To this purpose we will follow the lines of 
the proof of Theorem III.2 in [HI with the appropriate modifications. 

We first estimate the L^-norm of $ — We obtain, using (PU]) the 

upper bound 

<^>{s) - E{K)is)\ ds < a-^ j ^{y/an) dy j \^{s) - - y)\ ds < 

Ca~^ I V(y/an) |yr dy<0{l) < . 
We now bound from above the integral 

dsEi\KiXu...,Xn;s)-EiK)is)\) . 

By a well-known computation we have 

var(/i„(Xi, . . .,Xn;s)) < 

2 



J^E (ms - Xi)/an)) - i'nismijiis - Xe)/an)) - M^)) 



where ipnis) = K{tp((s — Xi)/an)))- Using Cauchy-Schwarz inequality and 
Hypothesis 15.11 as in the proof of Section El we get 

ds E . . . , s) - E{hn){s)\) < ■ 



in an 

Summarising we obtain 

E{K)<0{l){al+ ^ 



'n an 



The theorem now follows by Chebychev inequality and Devroye inequality. 
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For results on kernel density estimation in the context of piece-wise ex- 
panding maps on the interval, we refer to ^S] and references therein. 

We will prove in Appendix O that the class of dynamical systems con- 
sidered in that is the class introduced in jTH], satisfies (P^ . for ID 
systems. As explained at the end of the previous section, it also satisfies 
Hypothesis 15.11 Hence the theorem applies. This class includes quadratic 
maps for a set of parameter of positive Lebesgue measure jl8| . 

7 Shadowing and mismatch 

For a fixed integer n, let E he a measurable subset of M"*^. For a trajectory 
Yi, . . . ,Yn of length n of the process (Xk) which is outside E, how well can 
we approximate this trajectory by a trajectory (Xi, . . . ,Xn) of the process 
belonging to E7 We first start with a result about the average quality of 
this "shadowing". We will denote by 7^ the set of trajectories of length n 
of the process. 

Theorem 7.1. For any integer n, for any measurable subset E ofW^'^, with 
P((Xi, . . . , Xn) ^ E) > 0, the function ^ defined by 




satisfies for any t > the inequality 




where D > is the constant appearing in 



Proof. We first apply Devroye inequality Q to the function 





(30) 



n 



Chebychev inequality yields for any s > 




The function Ze is measurable, see [H| 
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Proceeding as in ^ and optimizing over s we obtain 

24/3 ^1/3 



ni/3p((Xi,...,X„) GE) 
The theorem follows using again Chebychev inequality. 



Remark 7.1. There is another way of estimating from above E{Ze)- For 
this observe that Ze vanishes in E. Therefore it follows from \Sn\) that 

p((Xi,...,x„) inzE)? <-■ 

n 

Hence 



VnP((Xi,...,X„) G^) 



We now derive a similar result for the number of mismatch at a given 
precision. 



Theorem 7.2. For any integer n, for any measurable subset E ofM. , with 
P((Xi, . . . , Xn) G E) > 0, and for any e > 0, the function defined by 

Z'E,,{Y,,...,Yr,) = ^ inf Card{l<i<n: ||X,-y,||>6} 

satisfies for any t > the following 

1 / 24/3zji/3 \ \ D 

^ y^E,e > ^2/3^1/3 y + ...,Xn)eE)) J - £2/3^1/3^2 

where D > is the constant appearing in (0). 



The industrious reader can follow the lines of the proof of Theorem 
IV. 2 in [Hj. Using Holder estimates instead of Lipschitz estimates yields the 
same formula with e^/^ replaced with e^''/^, for any < rj < 1. However 
the constant D depends on rj in an implicit way, so it is not clear how to 
optimize over ry. 

For the case of dynamical systems, given an initial condition y outside a 
measurable subset S of the phase space with positive measure, the questions 
considered above mean that we look how good is the shadowing of the orbit 
of y by an orbit starting from S (in that case E = S x M^""^)'^). 
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8 Almost-sure central limit theorem 

We say that the process (u(Xfc)), where u is a real-valued function, satisfies 
the Central Limit Theorem if 

g-'-'^j- ""'"'< A ^^/' e-er.^ ,31) 
where (u) is assumed to be strictly positive and is defined by 

oo 

= C{l) + 2j2C{i) (32) 

e=2 

where we assume that the series is finite (see (0} for the definition of C{i)). 

We will prove an Almost-sure Central Limit Theorem, see e.g. for 
a review of this field. Our result is slightly stronger since it asserts the 
convergence in the Kantorovich distance k already used above, see formula 
H27jl. We shall use it for measures on M and real- valued Lipschitz functions 
on R. Note that we can replace g hy g — g{0) in (|27|) since /xi and /i2 
are probability measures. In other words there is no loss of generality in 
assuming 

5 G ^0 := {5 e ^ I 9{0) = 0} . 

It is convenient to define the sequence of weighted empirical (random) mea- 
sures of the normalized partial sum Sk = u{Xi) -|- • • • -|- u{Xk) by 

1 " 1 

k=l 

where Dn = Ylk=i p shall investigate the convergence of this sequence 
of weighted empirical measures to the Gaussian measure in the Kantorovich 
metric. 

We now state the result of this section. 

Theorem 8.1. Consider the process (u(Xfc)) where u is a Holder contin- 
uous function with zero /x average (recall that fi is the common law of the 
Xf^'s). Assume that o"^ 7^ (see ^^^^ auto-covariance of {u{Xk)) 

is absolutely summable and that HSl]) holds (central limit theorem). Then 
¥-almost surely 

lim K(An,Af{0,cr^)) = (33) 

n— >oo 

where AA(0, cj^) is the Gaussian measure with mean zero and variance a'^ . 

The assumptions of the theorem hold for the class of dynamical systems 
discussed in ^HSj. For piece- wise expanding maps of the interval, a stronger 



hm P 

n— +00 
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result was proved in 4 . Notice that this theorem immediately implies that 
almost-surely An converges weakly to the Gaussian measure. 

Proof. We first prove that 

lim E(k(A,AA(0,cj^))) = 0. 

n—>oo 

Let B he a positive constant to be chosen large enough later on. We have 
for any g £ vanishing at and any x 

\g{x)\ < |x| . 

Therefore 



< sup / g(dAn-dM{0,(7^)]+ [ \y\dAniy)+ 

g&^oJ-B \ / J\y\>B 



\y\dM{Q,a'){y) . 

\>B 

(34) 

We first estimate the expectation of the second term uniformly in n. Since 
the correlations are absolutely summable, we get for any j 

ns]f'^<0{l)^j. (35) 

Therefore, using Cauchy-Schwarz and Bienayme-Chebychev inequalities we 
get 

^ (/.,>.|*^"<^>) - ) s ^ ■ 

(36) 

In order to estimate the first term on the rhs of (|34|) . we observe that since 
[—B^B] is compact, we can apply Ascoli-Arzela theorem to conclude that 
for any e > there is a number r = r(e) and a finite sequence gi, . . . ,gr 
of functions in such that for any g £ there is at least one integer 
1 < J < ^ such that 

sup \g{y) -gj{y)\ < e . 
\y\<B 

Therefore 

sup / g(dAn- dMiO,a^)] < sup / gj ( dAn - dM{0,(7'^)] + 2€ . 

gd^oJ-B V / l<j<r(e)J~B \ J 

(37) 

We now consider the r sequences of random variables 



Yn,j = j ^gj [dAn - dM{^, a2) j 



with 1 < i < r. 
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We first estimate the variance of Ynj- Let the sequence of functions 
{Knj) of n variables xi, . . . ,Xn and 1 < j < r be defined by 



K-ii.j (xi , . . . , X 



1 " 1 

fc=i 



9j 



where A^(0, a"^)(-) denotes the integration against the Gaussian measure. It 
is easy to verify that all these functions are separately Lipschitz with respect 
to all their variables, and that the Lipschitz constant with respect to the 
q^^ variable is bounded by 0{l)/{^/qDn) uniformly in n. Applying Devroye 
inequality (j3I) we get 



< 



0(1) "1 ^ 0(1) 

Dn 



We now have using Cauchy-Schwarz inequality 



E 



( sup Yn,i] < E j ^ \Yn,,\ I < E^ 
\l<j<r' / \j=l / j=l 



+ E 



E{Yr 



< 



rO{l) 



/Dr. 



E(y, 



< J^var(y„j)^/V^ E(y„,,- 

By the central limit theorem (|31() . we have, for each j 

lim E(y„j) = , 

n^oo 

and therefore, from the above estimates, for a fixed r(e) we have 

lim sup E ( sup f gj (dAn - dM (0, a"^)] ) < . 

It now follows from and (|57|) that for any e > and any i? > 
< limsupE(K(^„,AA(0,cj2))) < 2e + + / \y\ dJ\f{0,a'^){y) . 

n^oo -D -'|?/|>-B 

Letting B tend to infinity and e to zero we get 

lim E(k(A,AA(0,cj2))) = 0. 

n— +00 

We now estimate the variance of K{An,M{0,a'^)). Applying Devroye 
inequality Q as above to the function Kn of n variables xi, . . . ,Xn 



Kn(,Xi, . . . , Xf, 



(38) 
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1 



sup — — 7 - 



we get 



E 



1 2> 



«(A,Ar(0,(72)) -E(K(A,Ar(0,(72))) 



var(if„) 



< 



0{1) ^ 1 ^ 0(1) 



If for < /9 < 1 we define 
we conclude that 



nk = e 



< oo 



which impUes by the B. Levi's theorem that 

hm {K{An^,Af{0,a^)) -E[K{An^,,^f{0,(T'^))]) =0 P - almost surely . 

We now observe that if < n < n^+i we have 

\K{An,M{0,a'')) - K{An„M{0,a^))\ 



< 



Dr,. - D, 



1 " 1 

«;(Afc,Ar(0,a2))+sup — V - 
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The first term tends to zero almost surely by our previous estimates. We 
now prove that the second term tends to zero almost surely. We have 



1 " 1 



1 " 1 

<— y - 

3=nk+l 



^+Ar(0,c72)(|x| 



nk+1 



< 



1 ^ n 



+ AriO,a^){\x\ 



It follows easily from our choice of (nfe) that 



lim 

We now prove the almost sure convergence to zero of the sequence 



Dr,,. , 7-3/2 
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For this purpose we estimate the expectation of the square of T^. Using 
Cauchy-Schwarz inequahty and (|35|) we obtain 



(lognfe+i-logn,. + 0(l))2 



It follows at once that E(T^) is summable in k. The result now follows using 
B. Levi's theorem. The theorem is proved. 



Remarks. We note that the above proof also leads to an estimate on the 
probability that K{An,M{0,a'^)) is larger than some given number e > 0. 

For a dynamical system (O, /) it often occurs that the invariant measure 
is supported on an attractor which is a small subset of the phase space 
Jl. When there exists a SRB measure, one would like to have Theorem 
18.11 almost-surelv with respect to Lebesgue measure on il. Assuming that 
the stable foliation is absolutely continuous, and the forward contraction is 
uniform and exponential along local stable manifolds (see [E! for several 
examples), it is sufficient to prove that 

lim \Knix, fix), r~\x)) - Knix, fix), f''-\i))\ = 

n—>oo ' ' 

where is defined by (|nH|) and x, x belong to the same local stable manifold. 
This follows at once from the definition of Kn and the uniform exponential 
contraction along local stable manifolds. 



A About a trigonometric series 

For the convenience of the reader we prove in this appendix the following 
(probably well-known) result on trigonometric series for which we have not 
been able to locate a reference. 

Lemma A.l. We have the following 



sup 

meN,a;6[0,27r] 



E 

k=l 



sin kuj 



k 



< oo . 



Proof. First observe that it is enough to assume that ui G [0,7r]. Now we 
have 



E 

k=l 



sin koj 
k 



1 



1 



ds + c.c. 
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By an easy estimate, one gets 



sup 

meN,uje[0,7r] 



1 pims 

4' \ . ds 
1 - 



/•'^ . 1 
/ e'' - 
Jo 



ds 



IS 



< oo 



Finally, 



Jo 2is 
/■'^ sins ^ /"'^ sin(m + l)s 
Jo s Jo s 



ds + c.c. 



ds 



J 



{m+l)u! 



sms 



ds 



It is well-known that the modulus of this quantity is uniformly bounded in 
LO and m. ■ 



B On the uniform decay of correlations 

In this appendix we prove a general result on decay of correlations which 
may be useful in other contexts. Consider a dynamical system on a phase 
space Q given by a measurable map / from 0, to itself. Let fi be an ergodic 
invariant measure. The decay of correlations is often proved in the following 
form: There is a non increasing sequence (7^) and two Banach spaces ^1 
and ^2 of measurable functions on ft such that for any functions ipi G ^1 
and ip2 € ^2, there is a constant C^^^^^ such that for any integer n 



V'l ° /"^2 di2 - i^i djj. V2 di2 



^ Ci^i,ip2ln ■ (39) 

It is often useful to have some information on the constant C^j 1 ™ partic- 
ular if it can be bounded by a product of norms of the two functions (and a 
uniform constant). It turns out that this apparently stronger result follows 
from the previous estimate under the following natural assumptions. 

i) The constant functions belong to ^1. 

ii) The integration with respect to fi defines a continuous linear functional 
on ^i. 

in) The Koopman operator U (of composition with /) is continuous in 

iv) SS2 is contained in the dual of SS\ (duality with respect to the integra- 
tion by \x) with a topology at least as fine as the dual norm topology. 

As will become clear from the proof, the result below is due to the special 
form of the correlation integral. 
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Theorem B.l. Assume the above properties i-iv), and inequality hold. 
Then there exists a constant K such that for any integer n and any ipi £ c^i, 
^2 S '^2, we have 

< K\\^l;iUM2\\mln ■ (40) 

A frequent example is = L°°(r2, dfi) while is a space of more 
regular functions (functions of bounded variation, Lipschitz or Holder func- 
tions). In [IHI, ^1 = 0§2 is the space of Holder continuous functions. We 
give below a proof based on the principle of uniform boundedness. 

Proof. We first deal with the easy case where for some integer no we have 
7no ~ 0- -^o^ ^'^y > ''^0) using the identity ipi o = (^tpi o J"-"o^ q jno 
and Hi), we conclude that the correlation integral (left hand side of (jSHJ) is 
equal to zero for any ipi G and -02 £ ^2- On the other hand, it follows 
from iv) that there is a positive number Kq such that 

sup 

IIV'ilUi<l , IIV'2llsa2<l 
and ()4U() follows immediately with 

K = Ko sup ||C/"||,^,7-i . 

n ,-y„>0 

We now assume 7„ > for any integer n. We first control the dependence 
on tpi and for this purpose we first fix ip2 G ^2- We then define a sequence 
of non negative continuous functions {pt"^) on by 

We have obviously for any integer n and any ■i/'i , ip'i and belonging to 

Ppii^'i + V''/) < pfii^'i) +Pt'W) and pt'{^Pl) = pti-i'i) • 
It follows immediately from (|39)) that for each -(/'i S ^1 we have 

supp^2(-^^) < C'^i.V'a < 00 . 
n 

Therefore, we can apply the principle of uniform boundedness jlUl Theorem 
1.29 section III page 136] to conclude that there is a finite constant 
such that 

sup pt'ii'i) < D^2 ■ 
">IIV'i||«i<i 



ipi ° dfl- ipidfi ip2 dfl 



■01 ■02 dfl 



01 o /"02 dfl- il^idfi -02 dfl 
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In other words, for any integer n, for any -01 G and any 02 G ^2 we have 

< ^V2ll^llUi7n • (41) 

We shall now control the dependence in ^2- Let A = N x i?i where Bi 
is the closed unit ball of ^i. We define a family {q\)\,^A of continuous, 
non-negative functions of ^2 by 

9(n,^i)(V^2) = 7, 

We have immediately for any A G A and for any -02, "02 "^2 =^2 

9a (02 + 0'2) < 9a (-02) + 9a(V'2) and qx{iJ2) = q\{-i^2) ■ 
Moreover it follows from 1)41(1 that for any 02 G ^2 

supqx{'tp2) < D^,2 < 00 . 
AeA 

We can apply as above the principle of uniform boundedness to conclude 
that there is a finite constant K such that 

sup gA(V'2) < K 
agA,|1i/.2|U2<i 

which immediately implies (|4n|). ■ 

In the case where 7^ in (PU]) is summable and assumptions (i)-(iv) hold. 
Theorem IB. II implies Hypothesis 15.11 with ,^1 = =^2 being the space of rj- 
Holder continuous functions (0 < 77 < 1). 

C A property of the density of the invariant mea- 
sure for a class of ID maps 

The purpose of this section is to prove that property in Theorem 16. II is 
indeed valid for maps on the interval satisfying the axioms of TS'. In other 
words the density of the absolutely continuous invariant measure belongs to 
a Besov space (see for definitions). In particular, quadratic maps for a 
set of parameters of positive Lebesgue measure jTSj are included. We refer 
the reader to ^Sj (and [3]) for notations and properties of such dynamical 
systems and their associated tower maps. 

Recall that the density $ of the SRB measure reads ^1 ITH] 

^(2/) = Y.Y. ^kMXp(A,)iy) (42) 

j>l k=0 



01 O /"''02 dfi- IpldfJ. 02 dfj. 



V'l o /"V'2 d^- il^idfj. 0^2 dfi 



25 



where we set, for any k < Rj and for any y G f^{-^j 



akjiy) 



where ykj is the unique point in Aj satisfying f^{yki) = y, and (p is the 
density of the /-^-invariant measure. It is convenient to assume that a^j 
vanishes outside f^{Kj). 

We wiU use repeatedly the following properties coming from |18j : 

(i) There exists ^ > such that e^^^ < oo. 

(ii) There are constants C > and a G]0,1[ such that for all j and all 



k < Rj and any y, y' in / (Aj 



akjiy) 



o-kjiy') 



< c a'^y'y'^ . 



We recall that s{y,y') is the separation time of the orbits of y and y' , 
see [iHl; 



(iii) There exists a constant C > 1 such that for all j and all k < Rj and 
any y in f^{Aj) 

\A,\<akj{y)\f{Aj)\<C\A,\; 

(iv) Let B := ||/'||oo > 1- For all j and all k < Rj and any y in f^{Aj) 



akjiy) < CB 



Ri-k 



Property (i) follows from the exponential tail for Markovian return times. 
Property (ii) follows from the distortion bound in |18| . Property (iii) follows 
from (ii) and the fact that f^\Aj is a diffeomorphism and ip is bounded. 
Finally, property (iv) follows from (iii) and the fact that f^^~^{f^{Aj)) = A 
and f^^~^\f^{Aj) is a diffeomorphism. 

We will use the following lemma. 

Lemma C.l. There exists a constant C > such that for any measurable 
set j4 G M we have 

^l{A) < C m{Ay 

where m is Lebesgue measure and g = min{0/ log i?, 1} > (9 and B are 
defined in (i) and (iv), respectively) . 
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Proof. We have using Holder inequality withp = log -B/(log B—mm{6, log B}) 
and q = p/{p — ^) = Q^^ 



Ri~l 



j>\ k=0 



-Rj-i / . ^ i/p 

-;>1 A:=0 



Using (iii), (iv) and (i) this is bounded above by 

0{1) m(yl)^/'? E E l^jl B^^^-''^^P-^^/P < 0{1) m{A)r 

j>l k=0 

The lemma is proved. I 

The main result of this section is the following theorem. 

Theorem C.l. For an interval map satisfying hypotheses of for any 
positive T < min jlog a~-'^/(2 log |(min{l, 6*/ log there exists C > 

such that 

I my)-^y-6)\ dy<C\6\^ 

for any 5 €zM.. In other words, <I> belongs to the Besov space A^'°° (see ini). 
Proof. It is enough to consider < 6 < 1. We have 

J My)-^y-6)\ dy< 

E E / «fej(y)X/fe(A,)(y) -«fci(y-'^)X/fc(A,)(y-'^) • (43) 

j>l k=0 

For a fixed 6 > 0, we split the sum over j and k in (|43|) according to the 
condition 5 > |/'^(Aj)|/2 and the complementary condition. The first sum 
is bounded above by 

Rj-l 

2 E E l/'(A,)l = 

2 E E f ™P ^i^M \fH^j)\] ( sup akjiy)] IfHAjW 

|/fc{Aj)|<2i 
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< 0(1) l^il < 

j>l k=0 

where this last inequality follows from (i), (iii), (iv). 



(44) 



Now we turn to the second sum, namely the sum over the indices j, k 
satisfying |/^(Aj)| > 26. This sum is bounded above by 



j>l k=0 

|/fc(Aj.)|>2i 



+ 



Yl Y dy\akj{y)-akj{y-d)\Xfk(Aj)iy-^) 

j>l k=0 

|/'=(A^)|>2« 

Since f^{A.j) is an interval and \f^{Aj)\ > 26, we have 



dy akjiy) Xfk(A,)iy) - Xfk(A,)iy - ^) 



< 



(45) 



0(1) sup akj{y) dy Xfk(A^){y)-Xfk(A,)iy-^) 
(Aj) J 



< 



0(1) sup akjiy) 6. 



On the other hand, using (iii), the same integral is bounded above by 
2supygjfc(A^,) afcj(y) |/^(Aj)| < 0(1) |Aj|. Proceeding as in (jH]) we ob- 
tain for the sum over j and k the upper bound 0(1) 6'^ . We now estimate 
for each j and k the integral 



dy \akj{y) - akjiy - ^)\ X/fe(A,)(y - ^) = 
/ dy\akj{y)-akj{y-6)\Xfk(AAy-^)+ (46) 



/HA,) 



dy \akj{y) - akjiy - ^)\ X/fe(A,)(y - ^) 



(47) 



It is easy verify that the integral (|46j) can be bounded above like the integral 
For the integral (|^7|l we have the obvious upper bound 



2 sup akjiy) \fH^j)\ < Oil) \Aj\ 



(48) 
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Using (ii) this integral is also bounded above by 

/ dyak,{y) a<y'y-'^ Xj^aAv " ^) 

where s(y, y — 6) is the separation time of the orbits of y and y — 5. In order 
to estimate this integral we introduce a partition of f^{^j) into four subsets 
defined by 



B 



kj 



yefH^j) s{y,y-6)> 



T log 5 
log a 



Bl, = {y G f\^,) n CBl^r \R{r^y^y-'\y)) > o logr^} 

where is the Markovian return-time function defined in ^H], and a :- 
1/4 logs. 



{yG/'=(A,)n(Bi^.rnBi^. 



\f^y^y-^){y) - fiy'y-s)^y _ s)\ < ^| 

We will estimate the contribution of these four sequences of sets separately. 
We have obviously 



auj{y) dya^y^y-'^ < 5^ \f\K,)\ sup auj{y) 



and therefore we can bound the sum over k and j using (iii) and (i). 
To estimate the contribution of - we introduce the set 

I T log 1^ I 

L log a -I 

C:= U {y f{y)^{R>^ iog<^-n} ■ 

£=0 

From the invariance of the SRB measure /i and Lemma IC.ll we have 

< 

L logo 
Now observe that 



1 X 

M > a log rH < C m(R > a log 6"^ . 

log a 



which implies using (i) and Chebychev inequality that 

Ukjiy) dy < sup aujiy) p.{C) < 



1 A 

sup akjiy) m{R > a logr^)^ < 
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0{l) log(<5-i) sup akj{y) 
Using (iv) and interpolating with the bound (|48|) we get 

j>l k=0 -^^fcj 

We now treat the integral over the set B|j.. We define the sets 
and 

Lf7log5-iJ 

From the invariance of the SRB measure we get 

^(P) < a log (5-1 fiiVo). 

We now estimate 

A^(^o) = E E / "'^i^^) 
j>i fc=o JfH^,)nVo 

As we have done several times above, each integral in these sums has two 
bounds. From the definition of Dq we have 

/ akjiy) dy < 2\/5 #{i : Rj < cjlog^-^} sup akj{y) ■ 

J/'=(A,)nOo yefH^j) 

Since f^^ {Aj ) = A for all j > 1 we have for any j > 1 that 

B^^Ajl > |A| . 

Therefore since the Aj's are disjoint and their union is A we obtain for any 
integer q > 1 

#{j : Rj <q}<Bl. 

It follows using (iv) that 

/ ak,{y)dy<0{l)6'/''B^^'''\A,\. 

Therefore interpolating with the trivial bound (|48j) as before one gets 

/x(Po) < 0{1) ,5^/(4 log -B). 
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We now observe that 
which imphes using (ii) that 



/ ttkjiy) dy < sup akj{y) < 



0{1) sup ak,{y) log(ri)5^/^'°s^. 

3/6/'= (A,) 

Using (i) and (iv) and interpolating with the bound i|48j) we get 
EEVs -''Mdy<0{l) (logr^))^/'°^'^ 5^V4(iogB)^ 

j>l k=0 ^Ij 

Finahy if y G B^^. we have using B — ||/'||oo 

^ < \ f^y^y-^){y) - f<y^y-^){y -5)\<5 B'^y^y-^^ . 

This immediately implies that 

^(^'^-^)^-21^- 
Using this bound, properties (iii) and (i), we obtain 



j>l fc=0 ^fcj 



0(1) 5loga-l/(21ogB) ^ ^ |^^.| < 0(;l) ^loga-V(21ogB) 
j>l k=0 

This ends the proof of the theorem. 
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